Hitherto there was a video communication system with a feel of presence by satisfying the sensation of the existence of a person in front of oneself, by extracting, for example, a human figure from the image of the space scene of one's own side, and superposing and displaying this image, and a human image sent from the opposite side, and a virtual space image preliminarily stored for displaying commonly with the opposite side (Japanese Patent Publication No. 4-24914, Hypermedia system personal communication system; Fukuda, K., Tahara, T., Miyoshi, T.: “Hypermedia Personal Computer Communication System: Fujitsu Habitat,” FUJITSU Sci. Tech. J., 26, 3, pp. 196–206, October 1990; Nakamura: “Distributed cooperative job support by virtual reality sensation through network,” Audio-Visual Compound Information Processing Research Group, Information Processing Society of Japan, 1993). In the prior art, in particular, in order to synthesize the images, methods about increasing the speed and reducing the memory capacity have been invented (for example, Japanese Patent Publication No. 5-46592, Image synthesizing apparatus; Japanese Laid-open Patent No. 6-105226, Image synthesizing apparatus).
In the prior art, however, image synthesizing systems for synthesizing two-dimensional still pictures and three-dimensional CG data have been proposed, but nothing has been mentioned about a method of realizing a system for synthesizing and displaying by decoding (expanding) plural motion pictures or sounds simultaneously. In particular, in the terminal device for simultaneously decoding, synthesizing and displaying plural pictures and sounds, nothing has been mentioned about the reproducing method of picture and sound not leading to breakdown due to shortage of terminal capacity or fluctuation of processing capacity. In addition, nothing has been discussed about the method of decoding, synthesizing and displaying plural pictures depending on the accounting situation.
More specifically, nothing has been considered about:
(1) method of controlling plural pictures, sound information, information describing the relation of plural pictures and sounds, and information of processing result; and
(2) method about determining method of priority of decoding, synthesizing and displaying of plural pictures and sounds when the terminal processing state is overloaded, and also reproducing and accounting.
Further, in the environments of decoding, synthesizing and displaying plural pictures and sounds simultaneously, nothing has been considered about the method of controlling the quantity of coding by varying the method of compression of image depending on the state at the receiving terminal side or the priority of decoding, synthesizing, and displaying at the receiving terminal.